Focusing on the issue that the Kernelized Correlation Filter (KCF) tracking algorithm has poor performance in handling scale-variant target, a multi-scale tracking algorithm called Scale-KCF (SKCF) based on Correlation Filter (CF) and multi-scale image pyramid was proposed. Firstly, the occlusion status of the target was got through the response of the conventional KCF algorithm's classifier. The multi-scale image pyramid was built for the occluded target. Secondly, the scale information of the target was obtained by calculating the correlation filter's maximum response on the multi-scale image pyramid. Finally, the appearance model and the scale model of the target were updated with the fresh target. The experimental results on comparison with some state-of-the-art trackers such as Structured Output tracking with kernel (Struck), KCF, Tracking-Learning-Detection (TLD) and Multiple Instance Learning (MIL) demonstrate that the proposed tracker of SKCF achieves the best accuracy and overlap rate than other algorithms. Meanwhile, the proposed tracker can be widely used in target tracking and achieve high precise target tracking.